Information Geometry and Iterative Decoding

Motivated by the success of iterative decoding algorithms, we consider the general problem of making inferences based on observed data, which are known a-priori to satisfy multiple sets of constraints. We consider relative entropy minimisation and an iterative algorithm for its calculation, which exploits the presence of multiple constraints. Using these principles, optimal iterative decoders, with guaranteed monotone convergence will be described, followed by reduced-complexity sub-optimal decoders which may or may not converge. Also discussed is the use of iterative relative entropy minimisation for optimal inference on sets of data that are generated independently, but are not independent given an observation (e.g. joint channel estimation and data detection). It is shown that the general marginalised iterative relative entropy algorithm has xed points. Conditions for monotone convergence (to a xed point) of the marginalised iterative algorithm (i.e. turbo-style decoder) are derived.